posted 11-27-2012 05:43 PM         

*Mental health patients have a bill of rights.

*Consumers have a bill of rights.

*Credit card users have a bill of rights.

I'm sure there are others.

Should polygraph test-takers have a bill of rights?

If so, what should the bill contain? If not, why not?

posted 11-27-2012 06:30 PM            

It is called the authorization form that we use to obtain informed consent.

Some agencies already have something which they actually call a "bill of rights." It is the same thing, but the name punctuates our obligation to provide the examinee with sufficient information that voluntary participation is ensured by the fact that the examinee was fully informed of his or her rights during the exam.

I'll post the authorization form I use, in which describes both rights and voluntary participation without using the words "voluntary" or "rights."

The real issue is not "rights" but informed consent. Do we provide enough information that the examinee's participation can be construed as a personal choice made with awareness of what the procedure would require, what recourse the examinee has during and after the procedure, what risks or discomfort may be experienced during or after, and the results will be used.

In medicine, patients have the right to privacy - meaning their information is not uploaded to a central data-base for any other professional to view or use. Patients also have the right to know when they are used in research.

This gets complicated.

Personally, I am in favor of clear procedures for informed consent, but I don't like campy names like "bill of rights." There is s "Bill of Rights," but it is not a form we use during the polygraph test... matter of fact, I'm not even sure we are actually talking about "rights" in the God-given inalienable and constitutionally protected sense. Maybe we don't want to add this kind of drama to every polygraph. Lots of talk about rights could lead to lots of legal discussions about rights. The thing about legal discussions - lawyers love 'em, but somebody's got to pay for 'em.

Maybe good straightforward procedures for informed consent are sufficient.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-27-2012 07:03 PM            

Corey

posted 11-27-2012 08:23 PM         

Should the test-taker have the right to know how the test works? And I mean, really works?

Should the test-taker have the right to know what peer-review research "validates" the technique used in any given test?

Should the test-taker have the right to know the opinions of polygraph as expressed by the AMA, the (real) APA, and anti-polygraph sources?

Should the test-taker have the right to know the test-giver's precise qualifications, AND that test-giver's own actual in-the-field accuracy rate?

Should the test-taker have the right to know that (our) APA's meta-analysis contains no truly independent studies, blind studies or double-blind studies?

If not, why not?

I mean, a polygraph "test" is a legitimate, on-the-level scientific test, right?

Why would knowledge/opinion/bias on the part of the test-taker influence a scientific "test" like the polygraph?

Don Krapohl -- arguably the leading authoritative voice in all things polygraph -- likens the accuracy of specific-issue polygraph testing to that of film mammography. Would any of the points I've listed above impact a mammogram result? Why should the "scientific" polygraph be any different?

posted 11-28-2012 01:29 PM            

Dan, when you talk about "how it works? And [you] mean really works?" it will be important to be a little bit more clear about how you think it really works.

First lets be clear that when you talk about "should" you are referring to questions of professional ethics. It will be easier to engage this conversation from that perspective. Over-use of the word "should" leads to interpersonal resistance and reactivity. In Rational Emotive Therapy (Albert Ellis), which is somewhat similar to cognitive therapy, they would say this: "don't sh[ould] all over people...they don't like it when you sh[ould] on them."

You are raising important questions and discussion about ethics and informed consent.

So,

Most scientific tests will work regardless of whether someone knows how it works and regardless of whether someone "believes" in it or not.

There is evidence indicating that at least some polygraph techniques continue to work with people who know a lot about how the test works. Specifically, look at the DLST study in Iraq, in which the examinees were students with multiple weeks of exposure to polygraph training. And of course we have experience testing sex offenders, who have taken, 10, 20, 30, 40 or 50 or more polygraphs during years and years of probation and treatment.

There may be some complications, such as when the test depends on the test administrator witholding some information. This may be the case with some polygraph techniques (i.e., PLCs).

In the past many examiners were taught to manipulate the examinee to believe in the infalibility of the test. We seemed to believe that the test really worked because of this belief and because of our manipulation.

Experience with polygraph-savy sex offenders and with DLC questions seems to indicate that the exam does not depend on the examinee's naive confidence in the infalibility of the test, and does depend on manipulation as much as we had previously thought. Smart people since Reid and Abrams, and continuing on those from Utah, have been telling us for a while that we are probably not convincing anyone and nor do we need to convince anyone.

We were also taught to believe that the exam worked - examinee's react more strongly to RQs or CQs - because of fear of detection and consequences. This is also proving false.

The test seems to really work because of the basic principles of scientific testing (i.e, present a stimulus, measure the response, lather, rinse, repeat, aggregate the data, and compare the results to normative cutscores), and because of the basic principles of psychology and psychophysiology, (i.e., emotion, cognition, and behaviorally/experientially conditioned learning).

To the extent that your testing procedures and your believe in why the test works depends on manipulation then it might be considered unethical to provide information that interferes with your ability to to that. Keep in mind that I'm using the term unethical in a the very broad sense of ethical principles and not the declarative sense of ethical rules.

If the reason the test really works does not actually depend on manipulation then perhaps it is not unethical to discuss how the test really works.

I think as a general principle, there is no ethical requirement to fully explain to an examinee exactly how a test works.

Ethical professional practice requires that an examinee be provided sufficient information, including how the procedure or results may affect the examinee, to make an informed choice to consent to or refuse the examination.

Some of the answers to your questions about exactly how much information to provide may be grounded in declarative ethical statements regarding procedures for obtaining informed consent.

So, why would it be necessary to declare it a “right” to be provided all the information you mentioned? The only good reason would be this: if not providing the information would lead to a violation of rights via a lack of informed consent. Controversy and differences of opinion surrounding the polygraph are NOT secret. Neither is information about the polygraph.

For this reason, I will argue that not dumping all this info on people does not deprive anyone of any right and does not present a threat to the examinee's informed consent. Therefore, I think there should (like how that sounds) be no right to all this information at the time of the exam. Much of that information gets way too far into the weeds for most people. This does not mean that we should not provide information if requested. We should at least be prepared to account for ourselves with published evidence. We should provide the best information we have about what we know about accuracy of the test, and we should inform people of the non-injurious discomfort (without drama) of the arm cuff, and we should inform people of who gets the results and what those results might say (deceptive or told the truth). How the results are used by the referring professional is, in my view, a secondary, indirect, concern to our objective of informed consent, but we should be clear about who has access to the results, and the general type of decision the test results might influence (not determine). In general, the ethical function of tests is to give information. Decisions are made by people. However, some agencies may have critical mission priorities that prompt them to use results in ways that others will not. (just my .02).

And finally, some of your questions appear a little short-sighted regarding what is achievable in reality. For example: it would be impossible for any examiner to know their actual in-the-field accuracy rate unless it were possible to confirm every exam. Obviously this is impossible. So, if this is your requirement, then you have effectively painted yourself into a corner from which you have solution to exit.

As to the lack of independent double-blind studies in the APA meta-analysis: if your position is that we know nothing until we have the ultimate definitive and final answer, then you will be waiting a very long time because there are no ultimate, final and definitive answers. There is always more to learn. So, the real issue is whether we think the APA meta-analysis tells us something of reasonably important value regarding the range of accuracy that we might expect from our polygraph techniques. Whether there is an ethical requirement to fully discuss all the details of every single unsolved issue is doubtful.

So, the real issue is this: how much information is needed to obtain the examinee's informed consent?

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-28-2012 02:48 PM         

Whew, you were spinning that reply so fast I was gettin' dizzy...

You're generally regarded as the smartest guy in the room polygraph-wise, but when it comes to ethics, I dare say the jury is still out.

Let me explain.

You went on record in this forum saying a trick stim test is not an ethical violation. I think that's a problem. A big problem.

In that thread, when I took Brain Morris to task for what appears to be a trick stim test, you said it wasn't an ethical violation because nothing changed for the student volunteer.

You don't know that. Not in the least.

Maybe that college kid was a rabid polygraph skeptic. What if the "result" from a trick stim test suddenly made her a true believer in polygraph? Wouldn't that change things?

Moving along...

The APA's meta-analysis is akin to the Roman Catholic Church writing its own critique on that institution's involvement with child sexual abuse. Hardly objective.

Of course I'm aware that an examiner with any appreciable degree of real-world experience cannot know his own accuracy rate. Isn't there something fundamentally wrong with that picture?

It seems that ignorance on the part of the test-taker is the examiner's ally. While that may make for "good polygraph," it makes for lousy science.

Dan

posted 11-28-2012 04:13 PM            

Now who is spinning... I said that the trick stim test was not an ethical violation in that it did not violate an ethical rule. That is all. There is no doubt that there are larger ethical issues. The question you raised was whether this violated an ethical standard of the APA. That is what I responded to.

More spin: you neglect to acknowledge that ignorance has also been regarded as an asset to the truthful examinee.

The polygraph profession has been forced, as it were, to come to terms with the issue of knowledge re how the test works. Oddly, this does improve the scientific footing we stand on.

There are always preliminary concerns about the bias or objectivity of any scientific review. Look at OTA, and NRC ---> we complain that they are against the polygraph. Any concerns about bias and objectivity are preliminary only and should always be balanced against the details in the report.

It is possible that the Catholic Church could write a proper and effective self critique of the issue. Asking the question about possible bias is OK. Reaching a conclusion about bias is impossible - and represent another form of bias - without actually reading and responding to the content of the report.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-28-2012 08:23 PM         

Yeah, I forgot... Ignorance of how polygraph works is indeed an asset to the truth-teller.

Hey, that's a real confidence builder for the "test"!

Again, let's get real...

IMHO, the APA's meta-analysis is a wildly optimistic vision of polygraph's theoretical accuracy, architected by a cabal of individuals whose livelihoods depend on perpetuating the polygraph science myth.

The APA's meta-analysis ignores the realities of real-life polygraph, such as the influences of countermeasures, anxiety of guilt by association (or the Othello Factor), psychological differences of various ethnic and personality types, etc. -- not to mention examiner incompetence.

Regarding accuracy of the Utah technique: How many Utah field studies are contained in the meta-analysis? How many Utah field cases?

For a more realistic view of real-world polygraph accuracy, randomly select a group of examiners and have them embark on a 60 Minutes-style mission to identify the culprit in a non-existent crime.

I predict the results will be nothing close to the APA's lofty numbers.

posted 11-29-2012 08:26 AM         

quote:

---

You talk as if Don, Ray, and others are intentionally deceiving the masses for profit, mafia style, through promoting lies about polygraph accuracy.

---

Come to think of it, he is "Don" Krapohl. Hmm... Corey, you may be on to something! Minga!

There has been much hand wringing over the last few years over being taken seriously by the scientific community, and being left behind as the legitimate forensic sciences steadily steam toward uniform standards and thus, higher credibility.

The hand-wringers are desperate for polygraph to be included in that heady mix.

But how can we get there from here if all we have to offer is insider documents, produced by people with a vested interest in the polygraph industry? The meta-analysis is a study of past studies. Nothing new, or not much new, was done. There was no independent/disinterested oversight.

For starters, I'd like to see a big-name liberal university with a highly regarded psychology department run some polygraph studies.

I'd also like to see the APA take on the A-P challenge.

And it would be most helpful, from a public image standpoint, if elements of the APA leadership were not associated with the notions that 1) polygraph examiners are chosen by divine intervention, and 2) it's perfectly OK to deceive people by using a trick stim test.

The last guy to stir the pot center stage -- and brother, could he stir -- was "stat." He ended up bailing from polygraph.

However, I would not count on lightning striking twice here in PDD Fantasyland.

Dan

posted 11-29-2012 09:36 AM            

This is your work. Why are you so down on polygraph now??

posted 11-29-2012 09:54 AM         

I'm not down on polygraph, just down on professing polygraph as science.

I remain a proponent of polygraph as a utilitarian art, as in a craft.

In my view, the MQTZCT -- with its "complex" rules and its "fancy" questions -- is a higher form of the art.

Yes, you have to work harder with the Quadri-Track, but the payoff is less FPs. That's important to me.

[Please note that the last sentence is likely to serve as a dog whistle for Barry C. I think it's safe for me to say that you can expect him shortly.]

Dan

posted 11-29-2012 11:14 AM            

It is not pointless. If someone does not clarify this then story stands as told. The point is not to convince Dan or anyone. He will make his own conclusions, as will everyone else. Of course, some will simply follow, but they will follow what makes most sense. If you don't offer an alternative, then the only perspective is the only one told. Again the point is not to convince Dan. The point is to learn something from the conversation with Dan. Also, other may read and benefit from all this. It makes us smarter and more capable of responding to our critics.

I do not agree with Dan, but that does NOT mean he is stupid and may not mean he is completely wrong. There are some things we may actually agree on. In reality there are probably many similarities despite obvious differences between people. (That is what I am learning, about all kinds of people from different cultures, languages, ages, etc.).

We may not agree, but Dan seems to have an important role in the polygraph profession. He challenges our assumptions and forces us to question our knowledge and then learn more. Dan pulls no punches and tells us more or less exactly what some of our more intelligent and aggressive critics are going to say. This does not mean that he is correct.

For us, it is an opportunity to learn to respond to critics and to correct the errors in their assumptions. If we cannot correct their errors, if there is any truth to the criticism, then we are obligated to learn to correct the deficit.

Dan,

By 60 minutes I assume you are referring to either the episode in which three examiners appeared to have manipulated their result to correspond to the indications of the referral source, or the 2001 episode in which Doug Williams first coached and determined Diane Sawyer to have successfully used countermeasures to defeat his own attempt to test her?

I would certainly hope that examiners today, with hopefully more awareness of the need for sound procedures, numerical scoring, and normative data than 1986, might be more resistant to being manipulated into manipulating their results. Perhaps I am wrong. I prefer to assume that you are wrong and that the majority of professional examiners today do not simply make up test results for convenience.

In the time since 1986, the polygraph profession has come a long way in terms of our ability to tolerate and make use of QC review. We still have more to do, but QC was basically unheard of in the past. When I first suggested a QC or QA review in Colorado they said this: “What's QC?” Then an examiner yelled at me and threatened me. Examiners in the past seemed to have grown accustomed to having ultimate expert power and having their results accepted implicitly and without question. This was probably based on an assumption of perfect accuracy, as indicated in some past studies and literature reviews. BTW, We don't need to QC every test in order for QC to have a positive effect on test quality, any more than we actually need or want a policeman on each and every street to maintain law and order. Doing so would only produce a police-state and culture of mistrust. In some programs the stakes may be so high that QC for every exam is a good idea. For most exams, the practice of QC for any important exam (i.e., all court cases and any exam used to make a decision that will affect someone's rights, etc.)

If you believe 60 Minutes to be an accurate assessment of polygraph accuracy, do you include this information and accuracy information in your full-disclosure with examinees?

I think we stated in the report, though without the dramatic adjective "wildly," that it is possible that the present information may be an overestimation of test accuracy. We also indicated that among the most likely reasons for that might be related to problems with field sampling methods and the overuse of confession criterion.

One of the things we did in the meta-analysis was to finally impress on our own profession the issue of confidence intervals that describe what the evidence tells us about the possible range of accuracy. In all of our discussions we have emphasized the need to outgrow our naive hope to ever know the definitive accuracy of the test, and the need to advance ourselves professionally to the point at which we are comfortable discussing the lower limit of the confidence range and not only the mean score. I showed this at the APA conference in San Diego and other conferences, and most people seemed to understand and tolerate the issue without any violent reactions. Bill Teigen even asked a question about whether we should lead with this point in a courtroom discussion. I suggested not lead with this, but be prepared in case there is a smart person feeding questions to the opposing counsel.

We also stated in the report that the meta-analysis does not attempt to address complexities such as countermeasures or special populations. Additionally, we were very clear about the fact that the meta-analysis is regarding criterion accuracy only and not construct issues involving psychological mechanisms or examiner competence - though a number of included studies were conducted with student examiners.

So, it seems you are critical of the meta-analysis because it does not and cannot answer every single question about polygraph accuracy. No single report will ever do that. And it seems you are critical of the meta-analysis for reasons that are already described in the report as beyond the scope of the project.

You reject the basic information we have because we did not address all complex and esoteric questions. In fact we advised that additional work needs to be done. Does this mean that we presently know absolutely nothing? No. Does this mean that our present knowledge is worthless? No. It means only that we don't yet know everything we need to know.

And finally, you are critical of the meta-analysis because it was completed by people who work in the polygraph profession. Again, questioning the potential for bias is OK. In fact it is a good idea. What is not a good idea is to act as if the question is a conclusion without actually addressing the content. Until we discuss the content and details its just an ad hominem criticism.

And now, for your only meager content question:

Re: the details about the Utah studies... You could read the report and find the answer. One of the things you will find is that the field and lab studies on the Utah seemed to produce both results and numerical scores that did not differ significantly. Meaning that the issue of field and lab studies seems to be less important than you think. Attacking the Utah studies over this point is really a deflection from more important concerns.

To be sure, there are un-answered questions regarding countermeasures and special populations. And there is always a need for more information and more precise accuracy estimates. The meta-analysis is not intended to be the final answer. It is simply what we know at this time. Nothing more. You can check the facts for yourself and see if we have reached conclusions that are divergent from other systematic reviews that you consider to be more independent or less biased in favor of the polygraph.

And finally, I want to be clear that I appreciate your comments and criticisms. I think it is helpful to the profession to face these issues and continue to learn.

BTW, I assume that if you were truly down on polygraph as science then you would contact the editors of that journal and explain to them that you feel ethically obligated to withdraw the publication for that reason. To do otherwise seems like nothing more than either promoting the polygraph as a science or insulting the whole realm of science by propagandizing a publication that you do not believe in.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-29-2012 12:23 PM         

Thanks for valuing my gig as the pot-stirrer.

The polygraph community has nothing to fear from me. Polygraph is too big to fail.

quote:

---

BTW, I assume that if you were truly down on polygraph as science then you would contact the editors of that journal and explain to them that you feel ethically obligated to withdraw the publication for that reason. To do otherwise seems like nothing more than either promoting the polygraph as a science or insulting the whole realm of science by propagandizing a publication that you do not believe in.

---

Why do you frame this as an either/or proposition? Can there by no middle ground? Do people have to believe in the Golden Calf of Statistical Alchemy to be accepted into this after-school Science Club?

My sense is that people tend to look askance at the fine print, as it were, or other embedded qualifiers such as "countermeasures" and "special populations."

To clarify things, perhaps the meta-analysis' bottom line should be something like this:

In summary, properly administered CQT techniques are capable of accuracy rates of 89%, except in cases involving countermeasures, or when special populations are tested. In such instances, all bets are off!

I wonder how that would sell. Not so good?

BTW... I wonder what it takes to be classified as member a "special population." Be a hot-blooded hispanic? A guilt-plagued Roman Catholic? A skid-row drug abuser? A tremoring wino? A polygraph-savvy sociopathic skinner? Be (legitimately) medically compromised?

That could be a pretty wide swath of people.

Back to the fine print... It pays to read things closely. For example, I wonder how many of the folks who cite your recent research on the Backster method are aware that the study contains hardly any Backster charts?

Dan

posted 11-29-2012 02:00 PM            

quote:

---

Why do you frame this as an either/or proposition?

---

Sorry, my mistake. I was thinking that you might like false dichotomies.

like this...

quote:

---

In summary, properly administered CQT techniques are capable of accuracy rates of 89%, except in cases involving countermeasures, or when special populations are tested. In such instances, all bets are off!

---

In fact we do meet a lot of people with problems. It is well known that people with mental health and developmental problems are over-represented in prisons and jails and criminal populations. They seem to be capable of causing problems and getting themselves into trouble a little more easily than people without all those problems. Certainly, mental health is not a sufficient or definitive cause for crime, but the proportion of people with these problems is sometimes greater among criminals than among the non-criminal population.

Should try to pretend they are completely normal? Probably not.

Is the polygraph just as good with these people as with normal functioning folks? Maybe, maybe not. Until we study it we don't know. We do know they sometimes have different psychological and physiological response characteristics.

Should we assume we know absolutely nothing about how well the polygraph works with them? Again, probably not.

Do we presently have any research on this? Actually there is some, but not much.

Why have we not studied this more thoroughly. Well, we barely have the resources to research polygraph accuracy under normal circumstances. Understanding test performance under normal circumstances is fundamental to understanding test performance under exceptional circumstances. There is obviously more to do.

So what should we do? Nothing? That is irresponsible. Should we pretend there is no issue here at all? Again, irresponsible. Will false confidence or over-confidence solve this problem or insulate us from it? Probably not. Should we assume we cannot help? They we sure to be useless. Should assume that polygraph will be completely ineffective? There is no real reason to assume that.

Again, with the false dichotomies.

In reality the polygraph probably works with these people, but they do have recognizably different functional characteristics and we do not have enough research to support confident assertions that there is no difference. We also lack normative data with which to calculate error estimates (a Daubert requirement) for these persons.

Probably the best thing to do is to proceed with caution. Some people will be comfortable with this. However, if we are accustomed achieving our own comfort in posture of over-confidence, then admitting the need to be cautious may feel somewhat awkward and uncomfortable. But lets be real. Polygraph testing can probably be helpful even though it is not perfect, and it can still be helpful even though an examinee's circumstances may be more complicated than normal.

Of course, if we believe that someone is actually unsuitable for testing then we are ethically obligated not to test the person.

How do we decide who is normal functioning and who presents with exceptional circumstances? Good question. The only responsible answer is this: we learn to work with and communicate and understand the other professionals that actually make these decisions. It is not difficult, and there are only a handful of words and concepts to learn.

As for you questions about the Backster methods. Your ability to ask these question are evidence that we did a proper and responsible job describing both what we did and the present state of our published knowledge-base. I am confident that the data are indicating to us that there no great cause for concern that the Backster technique is less valid than others. If someone wants to argue that the well-described sampling limitations completely undermine anything we might possibly know about the Backster technique then I think there is a need for some description of exactly why - in terms of known constructs - it would be so very very different. Seems impossible to me, because the similarities are so obvious. But if you are not impressed and not convinced that is OK. Nobody is forcing you to believe anything or use any particular technique.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-29-2012 03:39 PM         

I get the sense that most practitioners in the polygraph rackets are minimalist when it comes to informed consent and examinee rights, at least in terms of polygraph's "big picture."

It makes sense, I suppose. An ignorant polygraph test subject is a good polygraph test subject.

Words to live by. Or to make a living by.

Let's return to the meta-analysis...

It still suffers from these four troubling perception points:

1. It's an insider document concocted by a cabal of operatives from a self-serving trade association.

2. The concoction's architects all derive their livelihoods from the polygraph industry.

3. The cabal's chief architect owns a polygraph school, is in wide demand as a polygraph consultant, and is a spokesmodel for the leading manufacturer of polygraph machines.

4. The meta-analysis was produced without independent or disinterested oversight.

I would think that all adds up to a pretty tough sell. But no one cares what a country bumpkin in New Hampshire thinks.

A more practical aspect is this: What do the old white guys in black robes think?

Has the APA's meta-analysis been instrumental in satisfying the Daubert standard in court?

That would be very telling. Anyone have any experience, or know of such success?

There was mention of an Alaska case on the A-P site not long ago. Polygraph was admitted (I think) despite arguments by Iacano.

UPDATE: Here's the blurb that appeared on the A-P site:

quote:

---

the debate Dr. Richardson seeks recently occurred in the Superior Court in Anchorage Alaska (3AN-09-11088CR) State vs Thomas Alexander and James Griffith...Polygraph was to be admitted in both trials.

---

Anyone know more about this?

Dan

posted 11-30-2012 02:30 AM            

Ray- your comment to me would be valid if this were Dan's first post insulting you and the APA. It's not. I and others have already chimed in. Also, I never called him "stupid" as you imply. I agree that we all learn by such debate and in no way suggest his objections be ignored. Yet there is a limit on how many times one can respond to the same old arguments.

Dan, I too wish there were polygraph research being conducted by disinterested third parties. However, that does not mean we should discount the research or insult the APA. I have read many articles by Ray , Don and others and have learned tremendously. They have advanced polygraph forward without any hidden agendas. I don't question their motives or integrity. However I see your point about perceived conflicts of interest.

Instead of us accepting the Anti challenge, our researchers should invite non- examiner psychologists, physiologists , criminal justice sociologists , and statisticians to join them during the studies from start to finish. Let them know we would like them to submit articles to their own publications, documenting any findings whether positive or negative for the polygraph profession. Invite respected criminologists to conduct a longitudinal study on the effectiveness of PCSOT and sex offender recidivism, deterrence etc. At the least , the APA/AAPP could invite those professors, psychologists, sociologists etc.. to annual seminars for free. Their peer review and other critiques will promote our profession so we need to work together on research.

posted 11-30-2012 08:57 AM         

Thanks for the post.

In this case, polygraph can come in if both defendants, in their respective trials, testify and are available for cross-examination.

That's a win.

Question: Could cross-examination open the door to lines of questioning about CMs?

It seems that could spawn further questions about how polygraph works.

The judges' 54-page decision admitting polygraph gets into some nuts and bolts, but I wonder if the jury will have an interest in knowing about ways to beat the test, among other polygraph-related issues...

Dan

posted 11-30-2012 01:09 PM            

My apologies. I did not mean to imply that you had ever called or thought of Dan as stupid. I think it is obvious he is not. My only point was to reiterate that I believe there is value in this discussion even though a lot of us disagree with Dan.

Dan can be insulting at times, but I'm a big boy and it's not really going to affect my low-self-esteem all that much. In reality, I've talked to Dan on the phone at least a couple times per year for many years, we email each other, and have even had dinner without any violence on a couple of occasions. Despite our differences, Dan and I have common interests in things outside of polygraph - shiny objects and loud things and such.

Glad you see Dan's point about perceived conflicts of interests. Also glad you see that our agenda is simply to help the polygraph profession. I do a lot of very difficult volunteer work, including training, conducting exams pro bono at times, testifying, research, writing, consultation and more. What I have learned is that it is a round-world, and it does come back in the end. So I am also able to do paid work. I am very very fortunate to have very interesting and important work to do, and opportunities to do it. In this profession I've met some of the most intelligent and decent people on earth who have contact with some of the most difficult, complex and important systemic problems anywhere - which also have very personal implications at the level of individual examinees and crime victims.

One of the things I know from working with sex offenders is that all they have to do is wear us out and they win.

So, sometimes there is no limit to the number of times we engage the same argument or same problem. Unless I am simply too tired or too distracted by some more interesting project or shiny object.

It's all good in the end. I have learned from everyone in this forum. You, Dan, Ted, Jim, Skip, and others.

We are looking for opportunities to study the outcome issues in PCSOT. As you can imagine, it is obsessive work that requires we engage the same conversations so many times that we get more efficient at identifying the real issues, get tired of the rut, and get focused on a viable solution. Simply laying down and conceding that nothing can be done is not satisfying. But it is like growing bamboo - not much happens for years, and then you get 20 feet of growth in a very short time. It takes planning of course, but also the right kinds of planning. It takes tolerating uncertainty and unanswered questions - and restraint from impulsively choosing short term answers simply to end the anxiety of uncertainty. The real answers will come if we do our work carefully.

Great ideas about inviting more academics. There is, of course, an important balance between hard-core geek stuff and practical continuing education for those of us who are in the trenches doing the really important day to day work.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-30-2012 01:52 PM            

quote:

---

4. The meta-analysis was produced without independent or disinterested oversight.

I would think that all adds up to a pretty tough sell. But no one cares what a country bumpkin in New Hampshire thinks.

A more practical aspect is this: What do the old white guys in black robes think?

Has the APA's meta-analysis been instrumental in satisfying the Daubert standard in court?

That would be very telling. Anyone have any experience, or know of such success?

There was mention of an Alaska case on the A-P site not long ago. Polygraph was admitted (I think) despite arguments by Iacano.

---

Here's what I know about the meta-analysis in Alaska (from talking to David Raskin):

posted 11-30-2012 03:55 PM            

The meta-analysis was discussed in a heard by an adminstrative law judge with the Colorado State Patrol and the State Attorney General. Testimony from opposing expert Alan Zelicoff was effectively blocked.

Charles Honts requested a copy of the report while he was preparing for testimony on a recent case.

Regarding Raskin's perspective that it is more of a literature review, I will of course disagree that it is a literature review. However, it is a little different than other meta-analysis in that the design of the analysis and report was centrally oriented around the question of named polygraph techniques that satisfy APA standard requirements for validation. We completed the analysis and attempted to answer those questions in a manner that employed and conforms to the methods and principles of meta-analysis. Hence the name: Meta-analytic Survey. There is no mystery around the fact that the study began as a survey task. One term for typical type of survey that would have been done is "literature review." So, I am not in complete disagreement with Raskin on this.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-30-2012 05:06 PM         

quote:

---

(The beauty of the NAS report and the meta-analysis is that their bottom lines on accuracy are pretty much the same. That speaks volumes....)

---

posted 11-30-2012 06:11 PM            

Corey

posted 12-01-2012 10:57 AM            

There are some ways in which it is rather in-convenient that we found nothing different than NRC 2003. At the time of publication we were all upset with the NRC because we thought they got it wrong. We had somehow been led to believe that the polygraph was much more accurate - near perfect even.

So I, for one, expected something a little different; though now I am not sure why. Then we realized that lots of literature reviews seemed to reach the same conclusion. And then we realized that John Reid and his associate in the 1970s actually did some of the most responsible work we have seen - telling us essentially the same result.

The most difficult thing about this is that we have either failed to improve or failed to demonstrate the improvement polygraph accuracy over time. We have actually improved a lot of things. But our impulse to over-estimate accuracy has confused outselves and caused distrust among others.

The most difficult things is that we believe we can have improved the polygraph. Why have we not shown and demonstrated improvemet? We have rules, associations, policies, standards, quality control, computers, algorithms and so on. Why has it not improved? Is it simply maxed-out? I don't think so. I think it is more likely that our improvements are still masked under past over-estimation. I also think it is possible that we have been overly focused on rules, politics, and personalities - and under-attentive to the importance of rigorously scrutinizing the evidence, emphasizing the use of proven methods, and forgoing the temptation to engage in excessive psychologizing and simply learn to do what the data tell us do to. When we do that the polygraph testing part is actually easier and we get to focus more of our energy on the useful information that we are pursuing.

But I don't think its all that convenient to have to acknowledge that we may have overestimated test accuracy. And its not all that convenient to take scientific stuff to people who are anti-science. And its not all that convenient to be scrutinized by smart opposing experts, and its not all that convenient to have to worry about whether smart guys like Raskin do or do not agree with our results.

Its very interesting, but its not very convenient. Even teaching this stuff is very interesting but not very convenient. It would help if I were a more attractive spokesmodel, or if I were funnier.

Dealing with the details and facts is sometimes more inconvenient than blowing sunshine at each other and playing personality games. It would be a lot more convenient to tell people the polygraph has near-perfect accuracy.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-01-2012 12:14 PM         

quote:

---

The most difficult thing is that we believe we can have improved the polygraph. Why have we not shown and demonstrated improvement? We have rules, associations, policies, standards, quality control, computers, algorithms and so on. Why has it not improved? Is it simply maxed-out?

---

The science, such as it is, is quite possibly maxed out.

But there are still ways to improve accuracy.

Perhaps you could start by embracing the much-discussed -- and condemned, by the Polygraph Scientologists -- fancy questions, complex scoring rules, and the art/craft/techne elements of polygraph.

What does polygraph have to lose? Credibility?

Dan

posted 12-01-2012 01:20 PM            

I would feel more confident with the conclusion that test accuracy is maxed-out if we had already attempted to optimize accuracy by questioning what actually works and what does not actually work, and then modify our testing procedures to more rigorously discard ideas that don't work and emphasize those that do. In many ways we have done this and it shows. But there are still some things that we tax our attention and energy with that probably do nothing but set the stage for pointless criticism among ourselves (e.g., proper use of symptomatics, time-bars, linear ratios, arbrary cutscores, sub-optimal decision rules, and circular conversations "linguistics" and whether the Federal ZCT is a single-issue or multi-facet test). Here is a challenge: somebody show us any evidence that there is a difference in terms of criterion accuracy between primary relevant, secondary relevant, strong relevant, weak relevant, evidence-connecting relevant, and guilty-knowledge relevant questions. My students had better learn these, 'cause if they mess it up somebody is gonna criticize them and call there test "invalid." Fancy ideas are abundant, but most of these hypothesis are probably going to fall apart when we actually study them. That is what happens. Most hypothesis do not work.

The Utah researchers have shown us that test actually does improve when we focus our attention on the things that actually work, and don't worry about things that do not make a difference.

In the meta-analysis we did acknowledge that there is some possibility that the 99%-ers are actually better. The problem is simply that the presently available studies are a little too confounded to confident assert that people should actually anticipate near-perfect accuracy when any random selected examiner uses those techniques (i.e., properly) on any randomly selected normal functioning person from any randomly selected neighborhood. Maybe accuracy would be near perfect. Maybe it would not. Near perfect accuracy would be great. We all want it. But science is generally about what happens for most people most of the time under most normal circumstances.

So, I am open the possibility of considering techniques that employ highly-psyhologized (almost to the point of mind-reading) questions), and complex scoring rules. Hey, published studies already show a .99 correlation coefficient between scores of two examiners. With near-perfect correspondence, we should have absolutely no concerns about the level of complexity as a limiting factor regarding test reliability.

Again I am open to this. But I will need your help, because i am not an expert with those techniques. The starting place is always a hypothesis that is translated into a research question.

Would you tell us if this is a reasonable hypothesis: techniques that make use of psychologically complex questions, and complex scoring rules are statistically significantly more accurate than other polygraph techniques?

The alternative hypothesis would be this: psychologically complex questions and complex scoring rules make no difference, therefore accuracy with these techniques would not differ signficantly from other techniques.

We need to find a geek to study this. I'm out. So are you. So is Matte. Nate Gordon also. Also exclude anyone who works with any of us. And reject anyone whose livelihood depends in any way on income from the polygraph profession. That eliminates all examiners anywhere. We are all biased. But complete independence is always the best thing right?

Who is left? Pretty much the only people would be academics. They like this kind of thing. But they require grant money, because that is how they survive: grant money benefits the Universities that pay their salaries that give them the leisure time to geek-out and obsess over scientific riddles. Often the story is this: no money no geeks. Some people do unfunded research (Rosky for example), but maybe that is another kind of motivation - bias. Others too.

Funding this is the best option: APA has a little research money, enough to keep a graduate student warm and living indoors, but no-where near the kind of budget that competent double-blind controlled field studies would require.

Most likely we'll have to go the un-funded route.

Know anybody with mad-geek-skillz, and the motivation, and sufficient knowledge about how to tackle both polygraph operation and polygraph research problems, who also has complete independence from any interests?

Let me know who it is and I will help the person to get started in any way I can.

Until then, we have only the few published studies. Nothing presently prevents anyone from using those techniques or from reaching their own informed conclusions about whether they feel confident in expecting the kind of results reported in those published studies.

All that is required is that we are familiar enough with scientific and research issues that we can both identify potential confounds, and evaluate the published factual evidence for ourselves. If we lack the skills or the courage to do this, then we will likely engage in discussions about personalities and characters (drama) instead of principles and evidence.

Again, I think it is too soon to conclude accuracy is maxed-out. We really have not done everything we can to question and correct our assumptions, focus on things that are proven and not frustrate our attention and energy with things that don't work as intended. But maybe we have also not done enough to investigate our more complex ideas. I hope you realize that I, for one, am not altogether opposed to complex ideas.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-01-2012 02:25 PM         

quote:

---

Funding this is the best option: APA has a little research money, enough to keep a graduate student warm and living indoors, but no-where near the kind of budget that competent double-blind controlled field studies would require.

Most likely we'll have to go the un-funded route.

Know anybody with mad-geek-skillz, and the motivation, and sufficient knowledge about how to tackle both polygraph operation and polygraph research problems, who also has complete independence from any interests?

Let me know who it is and I will help the person to get started in any way I can.

---

Why can't Uncle $ugar (DACA) do it?

Thanks to our tax dollars, they have the resources.

They'd have no dog in the fight, money-wise, advocacy-wise, or otherwise.

Or would they?

However, it would call for an about-face...

DACA led the jihad to dumb polygraph down. Maybe this perceived cap on accuracy is the price that had to be paid...

But perhaps a better question is this:

If accuracy is the goal, why wouldn't they want to do it?

Dan

posted 12-01-2012 05:57 PM         

quote:

---

The goal would be to create a simple, standardized testing procedure that ALL examiners would eventually follow if they want to be taken seriously. The "art" of polygraph should only be needed for rapport and interrogation skills. Those that follow traditional methods would certainly be allowed to- much like old school mystical chiropractics versus science based chiropractic practices. Their industry became very strong once science was allowed in the door.

---

Corey,

I think we're already there, at least in large part. The simple, standardized (dumbed down) methods claim accuracy of about 89%.

BTW, we're talking incident-specific tests, obviously.

Let's say the "science" of polygraph is really behind the current, generally acknowledged (by the indu$try, that is), accuracy rate of 89% (give or take a point).

Is it so far fetched that it might take a gifted examiner (an "artist"), using a "mystical" technique, to exact the remaining 11% out of the process?

If so, why?

Like it or not, there's an art/knack/gift/craft component to polygraph -- and not just the rapport and interrogation stages.

Chiropractic used to be regarded as voodoo medicine. It was banned in Massachusetts for decades, that's what so many practitioners set up shop in NH border towns on the MA state line. Nowadays, it's not only legal in the commonwealth, but many health plans cover chiropractic services. Why? It gets results, at least for many people. Is it quackery? Only if it doesn't work.

One of the several things Ray and I have in common is music. In fact, we both made a living, more or less, by playing gigs at one time or another. In music, you can have the finest instrument, the greatest teachers, all the practice, drive and dedication in the world, and never rise above a certain level. Or simply suck altogether, as a few of my tone-deaf students did. What's the difference? The gift. The knack. That innate trait that makes someone a "natural" at something.

Professional musicians have it (as do other artists), major league sports players have it, the best surgeons have it, and so on.

Why is it so hard to believe that the elusive 11% of polygraph's potential accuracy can be attributed to a "special population" of examiners using proprietary techniques?

Another example: Military combat outfits have "designated marksmen." These guys aren't necessarily snipers, at least not in the Carlos Hathcock tradition, but they're exceptional shots. That got me to thinking...

When I was in the army, there were a lot of hillbillies who were crack shots. That was to be expected. But there were "Leave It To Beaver" types of kids who had never touched a rifle until basic training who quickly developed into very fine marksmen, while others trainees remained dangerously dreadful shooters. Why was that? We all used the same type of rifle, had the same schooling from the same instructors, the same lots of ammo, fired for record in the same weather, etc. There was only one variable: the individual.

I believe there are similarities, which hinge on natural ability combined with advanced techniques, applicable to the polygraph field.

posted 12-01-2012 07:23 PM            

I'm afraid NCCA would be viewed by the purists as non-neutral. It has already been hinted at by the NRC. If they are not already working on this then I would assume that you and I are perhaps not fully understanding their mission. Polygraph may primarily be a tool for enhancing interview information for some, and may be a tool for amplifying the intensity or sensitivity of an investigation for others. the highest level of accuracy, given our current methods may include a balance of test sensitivity and test specificity that compromise the objective of achieving the highest sensitivity level possible. I really don't know. Just guessing.

clambrecht:

[quote]The hurdle that we admit is there is that there is no direct connection between deception and physiology. [quote]

I think what you mean is that there is no perfect correlation between any particular physiological reaction and deception. What we need to realize and remember is that this is true in ALL fields of testing it is not a deficiency unique to polygraph and there is not need to continuously emphasize and teach ourselves to harbor any sense of insecurity or inferiority over this.

Finding a unique physiological index is very very rare. Pregnancy tests are good example. The achieve high sensitivity and high specificity by identifying a hormone that is produced when a person is pregnant.

Then some clown (male) goes and pees on his girlfriends pregnancy test as a joke and gets a positive result. He posted it on Reddit.com and was advised that he has testicular cancer. So it turns out that the pregnancy hormone is not unique to pregnancy.

-- more later --

gotta get this plane...

r

Pr

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-01-2012 10:51 PM         

quote:

---

I'm afraid NCCA would be viewed by the purists as non-neutral. It has already been hinted at by the NRC. If they are not already working on this then I would assume that you and I are perhaps not fully understanding their mission. Polygraph may primarily be a tool for enhancing interview information for some, and may be a tool for amplifying the intensity or sensitivity of an investigation for others.

---

Ray,

You're right. Absolutely right. What was I thinking?

Safe travels,
Dan

posted 12-02-2012 12:46 PM            

All good points as usual.

But again I will disagree.

Expertise does matter. No question about it.

Not everyone can be a doctor, lawyer, tennis player or backstball star. Aptitude matters.

But good training, good education, and good techniques also matter.

Good technology matters too.

We have been emphasizing our wizardry and magical prowess for a long time. We will continue to do that.

But the missing piece is still generalizaility, and our ability to account for the worse-case-scenario or lower limits of what we know. Optimistic estimates are probably better for business. I get it. Its easier to sell 99% accurate polygraphs than 89% accurate polygraphs.

Try selling polygraph art to the US Congress in their legislative activities to regulate forensics.

Should we simply tell them to not bother including us because we are not scientists? We are artists. We are technicians. Would you have us take the position that we comfortable not being included in forensic science legislation?

Is there nothing reproducible about he polygraph? Do we simply have to wait to stumble across the next child-prodigy?

Probably not.

We can actually teach and reproduce some things and generalize some knowledge and skill.

If we neglect the science part, and focus only on the expertise part, then we will can anticipate someday being accused of an arcane posture of "trust me I'm an expert," and "I can't prove it with data and evidence, but I know I am right." Seems doubtful to me if that is going to be an easy sell.

.02

posted 12-10-2012 10:08 AM         

quote:

---

Dan, I too wish there were polygraph research being conducted by disinterested third parties. However, that does not mean we should discount the research or insult the APA.

---

Please cite my insult(s) to the APA.

posted 12-10-2012 09:17 PM            

:-)

.
.
.
.
.

posted 12-10-2012 09:27 PM         

Just copy and paste my insult(s) using the embedded UBB Code "quote" function.

But seriously, yes -- or retract your allegation.

Dan